Although the collection of survey data using the internet has become increasingly popular, the concern whether the responses are representative of the larger population has presented a major challenge to survey researchers. Using data collected from the Student Experience in the Research University in the Twenty-First Century (SERU21) study, this paper seeks to compare the characteristics of respondents and non-respondents and the methodologies to elicit responses from a well-defined undergraduate population at the University of California system.
About seven thousand undergraduates from seven participating University of California campuses were sampled. Electronic invitations were sent to each student to solicit participation in the study. Students were also randomly assigned to three different subgroups, each receiving a different email invitation sequence. The variation was designed to explore which strategy would provide the most effective mean to yield positive responses from students. In addition, a follow-up telephone survey was conducted with a representative sample of non-respondents after the completion of the online survey. The follow-up telephone survey of 216 non-respondents collected detailed information regarding the reasons for their lack of participation as well as selected items from the online survey.
Using information from various sources as described above, the study will: (1) compare the characteristics of respondents and non-respondents to see if response biases exist; (2) compare the response rates from the three groups to see which invitation sequence is the most effective method; and (3) examine whether non-responses may also be conditional on the invitation sequencing. Findings from the study will provide an informed understanding of possible biases in collecting survey data over the internet, even in a well-defined population that has wide electronic access.